Smart Eye's eye tracking software is part of its internal sensing system.
1.3 million people die from road traffic injuries every year. In response to this loss of life, automakers have implemented driver monitoring systems to improve road safety by understanding signs of driver damage (such as drowsiness or distraction). The adoption of driver monitoring systems is also being driven by regulatory frameworks such as the European New Car Assessment Program or Euro NCAP, which require the system to have a five-star safety rating.
However, the upcoming Euro NCAP regulations, the American Hot Car Act, and the American Infrastructure Act will soon require the system to not only measure the status of the driver, but also the entire cabin. This requires deeper insight than the driver monitoring system can provide.
Therefore, we are witnessing the evolution of driver monitoring systems to "internal sensing", which is a rapidly emerging market in which multiple methods such as artificial intelligence and computer vision are used to measure the status of the driver, cabin, and occupants. In addition to meeting regulatory requirements, these systems also provide automakers with excellent opportunities to differentiate in terms of safety and experience.
The internal sensing system encapsulates the key functions of the driver monitoring system. They monitor:
But to understand the full background of dangerous driving behavior, one needs to understand what is happening elsewhere in the cabin. Drivers can be distracted by many things, whether it's their cell phone or the crying baby in the back seat. Internal sensing provides context through occupancy detection (how many people are there, where they are sitting, and seat location); and activity detection, which identifies mobile phone use, eating, smoking, or other distracting activities.
Another key safety feature is the ability to detect children or pets left in the car-this is another cause of vehicle death. The internal sensing system uses occupant detection to identify children or object detection to identify child seats to solve this problem.
Object detection also has interesting applications for carpooling or other shared mobile environments. For example, the system can classify and detect when items such as luggage or mobile phones have been abandoned, thereby increasing the probability of returning them to the rightful owner.
Internal sensing also provides an opportunity to improve the mobile experience. Insights into occupancy rates and data collected by analyzing gazes, reactions, facial expressions, emotions, activities, and objects used by people will inform the status of the cabin, the driver, and the passengers. This people-oriented approach is very powerful. It will enable automakers to provide mobile experiences that, in addition to providing unparalleled safety for all vehicle occupants, can also improve comfort, health and entertainment.
For example, if the driver expresses frustration in traffic, the internal sensing solution can prompt the in-vehicle system to play soothing music, or suggest to stop for a break. Or, if the system recognizes a restless child in the back seat, it can recommend a movie to provide some much-needed entertainment.
As we reach higher and higher levels of autonomy, these applications will only become more attractive. When the mobile experience is no longer a driving behavior, consumers will not only choose their vehicle based on the operation of the vehicle-this is a big selling point for automakers considering how to differentiate and gain a competitive advantage.
Understanding the state of the entire cabin is complicated. It requires a multi-sensor approach—similar to what we see in external sensing systems, which have expanded from one or two sensors to include multiple cameras, radars, lidars, etc. Internal sensing will follow suit, using existing cameras for driver monitoring systems, as well as additional RBG and infrared cameras located throughout the cabin to provide an overall view under various lighting conditions.
The internal sensing system also requires advanced artificial intelligence software, which is built using deep learning and massive data. Deep learning models require a lot of data, and must use a lot of real-world data for development and training. Once trained, these systems can learn to recognize and track the status of the cabin and the occupants in the vehicle. From there, an additional layer of analysis can determine whether the occupants are enjoying their experience, whether they are interacting with objects, whether they are sitting in the correct position, whether they are wearing seat belts, and so on. Only in this way can the software understand what is happening in the vehicle and prompt relevant and safe vehicle adjustments, which can be applied to everyone without prejudice.
Nevertheless, with the technological advancement of automotive-grade hardware and software, it is time for automakers to develop—before regulatory requirements, competitors, and consumer expectations advance.
Have an opinion on this story? Click here to submit a letter to the editor and we may print it out.
Please enter a valid email address.
Please enter your email address.
Please select at least one newsletter subscription.
See more newsletter options on autonews.com/newsletters. You can unsubscribe at any time via the link in these emails. For more information, please refer to our privacy policy.
Sign up and send the best car news directly to your email inbox for free. Choose your news-we will provide it.
Get 24/7 in-depth, authoritative coverage of the automotive industry from a global team of reporters and editors covering news critical to your business.
The mission of Auto News is to be the main source of industry news, data and understanding for industry decision makers interested in North America.
1155 Gratiot Avenue, Detroit, Michigan 48207-2997
Automotive News ISSN 0005-1551 (print version) ISSN 1557-7686 (online version)
Fixed Operations Journal ISSN 2576-1064 (print) ISSN 2576-1072 (online)